.\"  -*- nroff -*-
.\"
.\" Logwarn - Utility for finding interesting messages in log files
.\"
.\" Copyright (C) 2010-2011 Archie L. Cobbs. All rights reserved.
.\"
.\" Licensed under the Apache License, Version 2.0 (the "License");
.\" you may not use this file except in compliance with the License.
.\" You may obtain a copy of the License at
.\"
.\"     http://www.apache.org/licenses/LICENSE-2.0
.\"
.\" Unless required by applicable law or agreed to in writing, software
.\" distributed under the License is distributed on an "AS IS" BASIS,
.\" WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.\" See the License for the specific language governing permissions and
.\" limitations under the License.
.\"
.\" $Id$
.\"
.Dd January 31, 2011
.Dt LOGWARN 1
.Os
.Sh NAME
.Nm logwarn
.Nd utility for finding interesting messages in log files
.Sh SYNOPSIS
.Nm logwarn
.Bk -words
.Op Fl ahnpqvz
.Op Fl d Ar dir | Fl f Ar file
.Op Fl m Ar firstpat
.Op Fl r Ar sufpat
.Ar logfile
.Ar [!]pattern ...
.Ek
.Pp
.Nm logwarn
.Bk -words
.Fl i
.Op Fl d Ar dir | Fl f Ar file
.Ar logfile
.Ek
.Sh DESCRIPTION
.Nm
searches for interesting messages in log files, where ``interesting'' is defined by an
user-supplied list of positive and negative extended regular expressions
provided on the command line.
.Pp
Each log message is compared against each
.Ar pattern
in the order given.
Negative patterns are specified with a ``!'' prefix.
If the log message matches a positive
.Ar pattern
before matching a negative
.Ar !pattern ,
or if none of the patterns match, then it's printed to standard output.
.Pp
.Nm
keeps track of its current position in the log file between invocations, so each matching line is only ever output once.
This information is kept in a separate `state file' (see the
.Fl d
and
.Fl f
flags below).
By default,
.Nm
interprets a missing state file as if the file were previously empty,
resulting in a scan of its entire contents; this behavior can be changed with the
.Fl a
and
.Fl i
flags.
.Pp
.Nm
will find messages in log files that have been rotated (and possibly compressed)
since the previous invocation, as long as the rotated file has a name equal to
.Ar logfile
followed by a suffix matching the rotated log file suffix pattern (see the
.Fl r
flag below).
.Pp
By default, each line in the log file is considered to be a separate log message.
Log messages spanning multiple lines are supported with the use of the
.Fl m
flag.
.Pp
If
.Ar logfile
is `-' then standard input is scanned.
In this mode, multiple invocations of
.Nm
can be pipelined together, optionally with intervening processing, to perform
transformations of the raw log input prior to matching.
Typically the
.Fl z
flag is used in this scenario.
.Sh OPTIONS
.Bl -tag -width Ds
.It Fl a
Auto-initialize when the state file does not exist.
This flag turns on the
.Fl i
intialization behavior in cases where
.Ar logfile
exists but the state file does not.
Normally this is not what you want, but this can be helpful in cases where it's important to
avoid a flood of repeated log messages caused by state files somehow disappearing between invocations.
.It Fl d
Specify
.Ar dir
as the directory in which
.Nm
will store state information between invocations.
When this flag is used, the name of the state file is automatically generated from
.Ar logfile .
.Pp
The default state directory is
.Pa @DEFAULT_CACHE_DIR@ .
.Pp
It is an error to use this flag and
.Fl f
at the same time.
.It Fl f
Specify the state file used to store state information between invocations.
Each
.Ar logfile
should have its own state file.
.Pp
It is an error to use this flag and
.Fl d
at the same time.
.It Fl h
Output help message and exit.
.It Fl i
Initialize the saved state for
.Ar logfile
as `up to date'.
This causes the next invocation to start its scan at the current
(as of this invocation) end of
.Ar logfile .
.It Fl m
Enable multi-line support using
.Ar firstpat
to match the first line of new log messages.
.Pp
Without this flag, each line in
.Ar logfile
is considered a separate log message.
With this flag, each line in
.Ar logfile
is matched against the extended regular expression
.Ar firstpat ;
non-matching lines are considered continuations of the previous line.
.Pp
Multi-line mode will work correctly even if a message crosses a log file rotation boundary.
.It Fl n
Normally, if the
.Ar logfile
does not exist,
.Nm
will exit with an error.
This flag causes
.Nm
to treat a non-existent
.Ar logfile
as if it were empty.
.It Fl p
Change default match behavior to non-matching.
By default, if a log message doesn't match any of the positive or negative patterns, it is considered a match.
This flag reverses this behavior so that these messages are considered non-matches.
.It Fl q
Disable the printing of matching log messages to standard output.
The process exit value can still be used to detect whether there were any matches (see below).
.It Fl r
Make
the extended regular expression
.Ar sufpat
the rotated log file suffix pattern.
.Pp
When
.Nm
detects that a log file has been rotated, it searches for the rotated log file by finding
files in the same directory that have the same name as
.Ar logfile
plus a suffix matching the rotated log file suffix pattern
(when multiple files match, the first one in sorting order is chosen).
.Pp
The default rotated log file suffix pattern is
.Pa ^(-[[:digit:]]{8}|\e\.[01])(\e\.(gz|bz2))?$
.It Fl v
Output version information and exit.
.It Fl z
Always start reading from the beginning of the file, even if the state file says otherwise.
This option is useful when reading from standard input.
.El
.Sh DETAILS
.Nm
treats missing state files as if the log file were previously empty, and it
never creates new state files for nonexistent log files.
The result is that when the
.Fl n
flag is used,
.Nm
correctly functions even if the log file doesn't come into existence until after the first few runs.
.Pp
Log file rotation is detected by comparing filesystem inode numbers, so
.Nm
may exhibit incorrect behavior if (for example) an existing log file is replaced by a copy of itself.
In this situation, use the
.Fl i
flag to reinitialize the saved state.
.Pp
Currently, the supported compression types for rotated files are
.Xr gzip 1
and
.Xr bzip2 1 .
The corresponding executables
.Xr gunzip 1
and
.Xr bunzip2 1
must be present on the user's
.Pa $PATH .
.Pp
When the
.Fl f
flag is not specified, the state files in the state directory have names created by taking the
.Ar logfile
from the command line and replacing ``/'' characters with ``_''.
Therefore, referring to the same log file using different pathnames can result in inconsistent behavior.
.Pp
The matching patterns are only applied to the first line of a multi-line log message.
However, if the first line is a match, then entire log message including all continuation lines will be output.
.Pp
In order to avoid the race condition where
.Nm
reads a partially written line, if the last line of
.Ar logfile
does not end in a newline character, it is not processed.
.Pp
The maximum supported length for a single line is 100,000 characters;
longer lines will be split and treated as multiple lines.
.Sh EXAMPLES
.Pp
.Bl -tag -width xxx
.It logwarn /var/log/warn
.Pp
Show all syslog warnings since the previous invocation.
.Pp
.It logwarn /var/log/warn | uniq -f 5 | logwarn -
.Pp
Same as above, compressing repeated instances of the same message into a single line of output.
.Pp
.It logwarn -p /var/log/apache2/access_log '^.* "[^"]+" 5[0-9]{2}'
.Pp
Show any Apache 5xx server errors since the previous invocation.
.It logwarn -p -m '^myprog: ' '!retrying' 'ERROR'
.Pp
Show lines not containing `retrying' but containing `ERROR', as well as any subsequent lines in a multi-line log message,
assuming the `myprog: ' prefix marks the start of each new log message.
.El
.Sh RETURN VALUES
.Nm
exits with one of the following values:
.Bl -tag -width Ds
.It 0
No matching log messages were found.
.It 1
One or more matching log messages were found.
.It 2
An error occurred.
.El
.Sh SEE ALSO
.Rs
.%T "Logwarn: Utility for finding interesting messages in log files"
.%O http://logwarn.googlecode.com/
.Re
.Sh AUTHOR
.An Archie L. Cobbs Aq archie@dellroad.org
